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CSJ ^ Abstract 

^->| We survey a few concentration inequalities for submodular and fractionally subadditive func- 

, ^ . tions of independent random variables, implied by the entropy method for self-bounding func- 

tions. The power of these concentration bounds is that they are dimension-free, in particular 
implying standard deviation 0{^^/E[f]) rather than 0{^/ri) which can be obtained for any 1- 
l ■ Lipschitz function of n variables. 

1 Introduction 

In this note, we survey several concentration bounds for submodular and fractionally subadditive 
jy^ ' functions of independent random variables. These bounds are obtained by the entropy method for 

O , self-bounding functions [H |31 UHl H] • This is a powerful technique developed over the last decade, 

which in particular recovers Talagrand's inequality [1]. We also recommend the lecture notes by 
Gabor Lugosi |5|. The connection between self-bounding and submodular functions is quite simple 
but perhaps not widely known. To our knowledge, the first application of self-bounding functions 
in computer science appeared in [7]. Similar concentration bounds for submodular functions have 
been proved recently by two sets of authors [3 [S] , unaware of the connection with self-bounding 
functions. Hence this note, which might be useful in applications involving submodular functions. 
Let us start with the definitions. 

vn 

^^ , Definition 1.1. A set function f : 2^ 



(N 
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• monotone, if f{A) < f{B) for all A C B C N . 

• submodular, if f{A U B) + f{A n B) < f{A) + f{B) for all A,B CN. 



^ ' • fractionally subadditive, if f(A) < "^ Pif{Bi) whenever Pi > and X^rass A > 1 Va G A. 

• subadditive, if f{A U B) < f{A) + f{B) for all AC BCN. 

Observe that the definition of fractional subadditivity implies that /(0) = (by taking A = 
Bi ~ % and /?i = or /3i = 2). It also implies monotonicity (by taking A C Bi and /3i = 1), and 
hence nonnegativity. The definition of subadditivity implies nonnegativity (by taking A — B), but 
not monotonicity. Submodularity implies neither non-negativity nor monotonicity. The property of 
being submodular is relevant for non-monotone functions (the cut function in a graph is an example) . 

We also use the notions of marginal values and Lipschitz functions. 
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Definition 1.2. For a function f : 2^ — > M, the marginal value of j E N with respect to A CZ N 
is fA{i) = /(^ U {j}) — f{A). f is called c-Lipschitz, if its marginal values are bounded by c in 
absolute value. 

A function is monotone if and only if its marginal values are always non- negative. Submodularity 
can be expressed equivalently by saying that marginal values /a(J) are non-increasing with respect 
to A. Furthermore, the following relationships are known |5]. 

Lemma 1.3. /// is non-negative monotone submodular, then it is fractionally subadditive. If f is 
fractionally subadditive, then it is also subadditive. 

These inclusions are strict, and there are simple examples separating the three classes [B]. Con- 
sider / : 2[3l -^ M+ such that /(0) = and f{S) = 1 whenever \S\ = 1 or 2. Then if / is submodular, 
we must have /([3]) < 1. If we define /([3]) = 3/2, / is not submodular but it is fractionally subad- 
ditive. If we define /([3]) = 2, / is not fractionally subadditive but it is still subadditive. Defining 
/([3]) > 2 would not make the function even subadditive. 

2 Self-bounding functions 

Self-bounding functions were introduced by Boucheron, Lugosi and Massart [2]. Self-bounding func- 
tions are defined more generally on product spaces; here we restrict our attention to the hypercube 
{0, 1}". We identify functions on {0, 1}" with set functions on A^ = [n] in a natural way. 

Definition 2.1. A function f : {0, 1}" — > K. is self- bounding, if there are functions fi : {0, 1}"^^ — > 
M such that if we denote a;*-*' — {xi, . . . , Xi-i, Xi+i, . . . , x„), then for all x and i, 

0</(.t)-/,(x«)<1 
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A typical choice of fi is fi{x^^^) — miua,; fix), which for monotone functions means fi{x'^^^) = 
f{xi, . . . ,Xi-i,0,Xi-i-i, . . . ,Xn). First, we show that fractionally subadditive functions are self- 
bounding. Hence, every non-negative monotone submodular function is also self-bounding. 

Lemma 2.2. Every fractionally subadditive function f : 2^ — > M+ with marginal values in [0, 1] is 
self-bounding. 

Proof. We use /^(a:^*^) = f{xi, . . . ,Xi-i,0,Xi+i, . . . ,Xn), which means that f{x) — fi{x^^^) is the 
marginal value of i with respect to a;'^*^. Thus the condition < f{x) — fi{x^^^) < 1 is satisfied by 
assumption. 

For a given x G {0, 1}", define A — {j : Xj = 1}, Bi = A\ {i} and /3i = ^^. Slightly abusing 

notation, we have f{x) — f{A) and /i(a;^*^) — f{Bi). We also have X^rieS; Pi — [n— 1);^ = 1 for 
each i £ A. Therefore, the definition of fractional subadditivity implies that 

n -. n 

fix) = f{A) <Y,(3J{B,) = -^/,:(xW). 

i=\ i=l 

This proves the condition J2^i^iif{x) — /i(a;'*')) < fix). D 



McDiarmid and Reed [TO] further refined the notion of self-bounding functions as follows. 

Definition 2.3. A function f : {0, f}" — > M is (a, b)- self-hounding, if there are a,b >0 and functions 
fi : {0, 1}"^^^ — > M such that if we denote x^^' — (xi, . . . , Xj_i, Xi^i, . . . , x^), then for all x and i, 

0</(a;)-/,(x«)<l 

and 

J2{f{x)-f.{x^'^))<af{x)+b. 

We remark that subadditive functions are not always self-bounding, or even (a, &)-self-bounding 
for any constant a, b. (See the example at the end of Section [3l) However, the notion of (a, &)- 
self-bounding functions is useful for us, because non-monotone submodular functions turn out to be 
(2, 0)-sclf-bounding. 

Lemma 2.4. Every 1-Lipschitz submodular function f : 2^ — > M+ is {2,0)-self-bounding. 

Proof. We consider / as a function on {0, 1}", and define fi{x^^^) = miuj,. f{x). Note that here, it is 
not always the case that /^(a;'-*'') is obtained by setting Xi = 0. Denote by A the indices i where the 
minimum is attained for Xi = 0, and by B the indices i where the minimum is attained for Xi = 1. 
(In case of equality, say we assign the index to A.) In both cases, we have < f{x) — fi{x'^^^) < 1 
since the marginal values of / are bounded by 1 in absolute value. 

We bound the sum of the marginal values in two steps. First, let us add up over all indices in 
A = {ai, . . . , Ok}- We denote by x°{A') the point where the coordinates on A' have been set to zero. 
By submodularity, we have 

k 
ieA j=l 

< ^(/(x°({ai, . . . , a,_i})) - /(x°({ai, . . . , a,}))) 

= f{x)-f{x"{A))<f{x) 

using f{x'^{A)) > 0. Similarly, we add up over the indices B = {&i, . . . , 6^} where the minimum is 
attained by setting Xi = 1. We denote by x^{B') the point where the coordinates on B' have been 
set to 1. By submodularity, we have 

e 
^(/(x)-/,(xW)) = ^(/(x)-/(xi({&,}))) 

e 

= f{x)-f{x\B))<f{x). 
Since (A, B) is a partition of [n], we conclude: 

n 

E(/(x) - /,(:rW)) = E(/(x) - /,(a;W)) + ^(/(x) - /.(x^)) < 2/(x). 
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We remark that the factor of 2 is necessary. For example, if 71 = 2 and f{xi,X2) = a;i(l — X2) 
(the cut function of one directed edge), we have /(1,0) = 1 but flipping each coordinate decreases 
the value by 1, which means J27=i(fi-'') ~ fii^^^^)) — 2- 

3 Concentration of submodular and fractionally subadditive 
functions 

Boucheron, Lugosi and Massart proved that self-bounding functions of independent random variables 
are strongly concentrated, using the entropy method [2]. They proved the following bound on the 
exponential moment of a self-bounding function. 

Theorem 3.1. If Z = f{Xi, . . . ,Xn) where Xi G {0, 1} are independently random and f is self- 
bounding, then for any A G M 

logE[e^(^-^[^l)] < (e^ - A - 1)E[Z]. 

By Leinma [2.2l and Lemma ll.3[ this bound also holds for any non-negative monotone submodular 
or fractionally subadditive function with marginal values in [0, 1]. Positive and negative choices of A 
yield Chernoff-type bounds for the upper and lower tails. We pick A = ln(l -I- S) for the upper tail, 
and A = ln(l — 6) for the lower tail. Applying Markov's inequality to the exponential moment, we 
obtain tail estimates similar to Chernoff-Hoeffding bounds. 

Corollary 3.2. If Z = f{Xi, . . . ,X„) where Xi G {0, 1} are independently random and f is self- 
bounding (or in particular, non-negative submodular or fractionally subadditive with marginal values 
in [0, 1]), then for any S > 0, 

/ s \ EfZl 

. Pr[Z>(l + 5)E[Z]]< 7^-1^ 
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• Pr[Z < (1 - 6)E[Z]] < e-'5'E[z]/2^ 

The power of these concentration bounds is that they are dimension-free, i.e. independent of n. 
In particular, they imply that Z is concentrated around E[Z] with standard deviation 0{y/E\Z]). 
Weaker bounds with standard deviation 0{y^) can be obtained by martingale arguments for any 
1-Lipschitz function. 

We remark that the tail estimates are often presented in a somewhat different form. In particular, 
[4] presents the upper-tail bound as follows: Pr[Z > E[Z] -I- i] < e~*'/(2E[z]+2t/3)_ r^j^jg bound is 
easier to work with, but it becomes weaker for large t. In particular, if E[Z] = 1, then it would seem 
that we need t = D,(\ogn) to make the probability polynomially small in n, while Corollarv l3.2l imDlies 
that t = S = r2(logn/ loglog?i) is sufficient. The difference can be crucial in some applications. 

For non-monotone submodular functions, we need to use more general bounds for (a, 6)-self- 
bounding functions, which were proved in |10j and strengthened in [?. 

Theorem 3.3. If Z — f{Xi, . . . , X„) where Xi G {0, 1} are independently random and f is (a, b)- 
self-bounding, a > 1/3 and c — (3a — l)/6, then 

• for any t > 0, Pr[Z > E[Z] + t] < e-5*V(aE[z]+6+ct)^ 

• for anyO<t< E[Z], Pr[Z < E[Z] - t] < e-5t'/("E[z]+6)_ 



Since non-monotone submodular functions are (2, 0)-self-bounding fLemma 12. 4p . we use this 
bound with a = 2 and 6 = 0, i.e. c = 5/6. We also substitute t = dE[Z]. 

Corollary 3.4. If Z = f{Xi, . . . ,X„) where Xi g {0, 1} are independently random and f is non- 
negative submodular with marginal values in [—1,1], then for any 6 > 0, 

• Pt[Z > (1 + 6)E[Z]] < e-^'E[z]/(4+55/3)^ 

• Pr[Z < (1 - 5)E[Z]] < e-'5'E[z]/4^ 

Observe that here, the upper tail decays only as a simple exponential for 6 — >■ cx), i.e. it is slightly 
weaker than the Chernoff-type bound in Corollarv l3.2l We do not know whether this is necessary. 

4 Subadditive functions 

Finally, we show that subadditive functions are not (a, 6)-self-bounding for any fixed a,b > and in 
fact do not enjoy concentration properties similar to Corollaries 13.21 [?^ 

A counterexample. 

. f{S) = \S\ for |5| < V^, 

• f{S) = V^ for V^ < 1^1 < (71 - V^)/2, 

• fiS) = V^ + |5| - (n - ^)/2 for (n - ^)/2 < \S\ < {n + V^)/2, 

• f{S) = 2^^ for |5| >{n + V^)/2. 

Clearly, the marginal values are in [0,1]. We claim that this function is subadditive. Consider 
f{A) + f{B): If both |A|, \B\ > y/^, then we have f{A) + f{B) > 2^" > f{A U B). So assume 
\A\ < 01. Then f{A) + f{B) = \A\ + f{B) > f{A\JB), because the marginal values are at most 1 
and hence f{A U B) — f{B) cannot be more than \A\. 

This function is not (a, 6)-self-bounding for any constant a, b: f{S) = |\/^ for \S\ = n/2 (assum- 
ing n even), and we get f{S)-f{S\{i}) = 1 for ah i G S. Therefore, T,7=iifiS)-f{S\{i})) = n/2. 

Indeed, this function is not sharply concentrated. Consider a uniformly random set R (corre- 
sponding to independent random unbiased variables Xi G {0, 1}). We have |_R| < (n — \/n)/2 with 
constant probability (roughly 1/V27re ~ 0.24, from the central limit theorem), and the same holds 
for |i?| > {n + Y^)/2. Therefore, with constant probabilities, f{R) is either ^/n or 2y/n, and the 
expectation is roughly |-\/n. The standard deviation is on the order of ^/n - such a bound can be 
obtained for any 1-Lipschitz function. 

Still, subadditive functions satisfy some concentration properties that do not hold for arbitrary 
1-Lipschitz functions. In particular, it is very improbable that a subadditive function attains a value 
significantly above 3 times its median. The following more general inequality is shown in [llj (see 
Corollary 12, which applies more generally to product spaces). 

Theorem 4.1. Let Z — f{Xi, . . . ,Xn) where f is non-negative subadditive with marginal values in 
[0, 1], and Xi e {0, 1} are independently random. Then for any a > 0,1 < k < n and integer q > 18, 

Pr[Z >{q + l)a + k] < Pr[Z < a]"«q-'=. 

In particular, if a is the median of Z and g = 2, we get Pr[Z > 3a -I- fc] < 2^^''. Of course this 
is not true for an arbitrary non-negative monotone 1-Lipschitz function. Let for example f{S) — 
max{0, \S\ — n/2}, and Z = f{R) where R is uniformly random. Then the median of Z is 0, but 
Z > ^/n with constant probability. 
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